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ABSTRACT 

To overcome various reporting constraints, numerous 
strategies have been developed to facilitate accurate and timely 
reporting of data concerning the impact of Louisiana's State-Funded 
Compensatory/Remedial Program. These strategies include: (1) using 
existing statewide communication networks; (2) employing the Basic 
Skills Test (BST) as the major data collection instrument; (3) 
identifying each student's skill deficiencies through the Student 
Profile Sheets; and (4) determining the impact of the program on 
students who are promoted through the Compensatory Education Tests. 
The evaluation results for the program were reported through the use 
of a modular format. The evaluation report drew primarily on data 
from the 1982, 1983, and 1984 administrations of the BST to provide 
longitudinal information concerning the effects of the program on 
student performance. As a result of this evaluation, two 
recommendations were made. The data supported the program on 
identified deficit skills but suggested that this focus be broadened 
to address additional skills prerequisite to the next grade level 
BST. Secondly, school systems were urged to consider carefully the 
decision to promote students who fail to attain the minimum 
performance standard on a specific BST; retention at the earliest 
level at which deficiencies are identified was strongly recommended. 



******************************* **************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
********************************************************** *****^^^^^ 

ERIC 



\ 



EVALUATION STRATEGIES: LOUISIANA'S STATE-FUNDED 
COMPENSATORY/REMEDIAL PROGRAM 



00 

o 
rv 

o 

UJ 



Janella Rachal 



Bureau of Evaluation 
Office of Research and Development 



A paper presented at the annual meeting of the Evaluation 
Research Society, San Francisco, California 
October 11, 1984 



LOUISIANA DEPARTMENT OF EDUCATION 



This public document was published at a total cost of $23.85; 25 
copies of this public document were published in this first printing 
at a cost of $23.85. The total cost of all printings of this docu- 
ment, including reprints, is $23.85. This document was published by 
the Louisiana Department of Education, P. 0. Sox 94064, Baton Rouge, 
Louisiana 70804-9064, to provide leadership for the continuous 
development, coordination, and improvement of education on a statewide 
basis under authority of Louisiana R.S. 17:21. This material was 
printed in accordance with the standards for printing in accordance 
with standards for printing by state agencies established pursuant to 
R.S. 43:31. 



4 

v3 



Louisiana State Department of Education 
P. 0. Box 94064, Baton Rouge, LA 70804-9064 
THOMAS G. CLAUSEN, SUPERINTENDENT 



ERIC 



U.8. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 
EDUCATIONAL RESOURCES INFORMATION 

CENT EM (ERIC) 
£^h» document has been reproduced as 
f received from the person or organization 

Originating it, 
D Minor changes have been made to improve 
reproduction quality 

• Points of view or opinions stated in this docu- 
ment do not necessanly represent official NIE 
position or policy. 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



INTRODUCTION 



The Louisiana Competency-Based Education Program is based on 
three major pieces of legislation. The first provided for the 
development of State minimum standards in reading, writing, and 
mathematics for all grades, kindergarten through grade 12. These 
standards were developed over a number of years by committees of 
teachers, curriculum specialists, central office staff, principals, 
and other education personnel at the State and local levels. Both the 
standards and their companion curriculum guides are in place in 
Louisiana's public schools. 

The second piece of legislation established the State Basic 
Skills Testing Program to measure student attainment of the minimum 
standards. The first Basic Skills Test (BST) was administered in the 
spring of 1982 to all grade 2 public school students who were addres- 
sing State minimum standards in language arts and mathematics. One 
grade level has been added to the program each year so that in the 
spring of 1985 grades 2-5 will be tested. 

The third major piece of legislation in the Competency-Based 
Program established the State-Funded Compensatory/Remedial Program to 
address skills . identified as deficient on the BST. Services were 
first provided during the 1982 summer session and the 1982-83 regular 
school year for students who qualified by scoring below the State 
Board adopted minimum performance standard on the Grade 2 BST. The 
program has expanded each year so that 1984-85 services include 
remediation in skills at grades 2-4. 



COMPENSATORY/ REMEDIAL PROGRAM BACKGROUND 
The Louisiana State-Funded Compensatory/Remedial Program is 
administered by the Department of Education, Bureau of Elementary 
Education. State funds are allocated to local school systems on a per 
remediation unit basis for those students who score below the State 
Board adopted minimum performance standard on each grade level BST. 
Local systems must serve eligible students during the summer session 
and/or the regular school year immediately following the spring 
administration of the BST. Both State and local evaluations of the 
program are mandated by the legislation that established the program. 



EVALUATION REPORTING CONSTRAINTS 

The purpose of the State level evaluation of the Compensatory/ 
Remedial Program is to provide information to policy makers in the 
Louisiana Legislature, the State Board of Elementary and Secondary 
Education, and the State Department of Education about the impact of 
the program on education in Louisiana. In order for this imormation 
to be most useful, it must be made available within the stringent 
time frame under which the varied audiences operate. Over the years 
several constraints have had to be overcome in order to maximize the 
usefulness of this evaluative information. 

One major obstacle encountered in the evaluation of the program 
has been the extreme difficulty associated with the reporting of 
complete census data concerning all students who qualify for the 
program. The logistics are becoming increasingly complex each year 
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with the expansion of the program to include additional grade levels. 
The program has grown from approximately 6,000 students served in 
1982-83 to 25,500 in 1984-85. Numerous program characteristics have 
also contributed to the complexity of the data collection needed for 
reporting this kind of information. For example, students qualify for 
the program on the basis of their language arts and mathematics scores 
on the spring administration of the BST, but do not receive services 
unti 1 the summer and/or regular year session. Based on criteria 
specified in local school system Pupil Progression Plans, these 
students could be retained at the level tested or promoted to the next 
level for the subsequent school year during which services must be 
provided. These factors along with others associated with the 
mobility of eligible students have made the tracking of student 
progress from qualification through program completion extremely 
difficult, and in many cases, impossible. 

The promotion/retention option among eligible students has placed 
an additional constraint on the evaluation in that it has limited the 
availability of pretreatment-post treatment information on promoted 
students. Retained students are the only ones to take the same BST in 
successive years. Thus, other means of assessing the impact of the 
program on the performance of promoted students have had to be de- 
veloped. 

Conflicting Legislative and school calendars have created ad- 
ditional data collection and reporting constraints. The education and 
general appropriation committees of the Louisiana Legislature convene 
in March and April of each year with the general session running from 



May through July. In order to be most useful, information concerning 
the impact of the compensatory education program must be made availa- 
ble to the committees during the same time period (March/April) during 
which the best indicator of that impact (the BST) is being admini- 
stered. The results of that BST administration are generally not 
available until July/August, in other words, after the close of the 
session. Again, other means have had to be developed in order 'co 
provide the Legislature with the best information available at the 
time when it is most needed. 



EVALUATION STRATEGIES 

In order to overcome the various reporting constraints described, 
numerous strategies have been developed to facilitate the accurate and 
timely reporting of data concerning the impact of the State-Funded 
Compensatory/Remedial Program. A concerted effort has been made to 
utilize existing statewide communication networks in an attempt to 
minimize the disruptive effects of extensive data collection on local 
school systems. Much of the requested information concerning the 
program has been channeled through local system evaluators responsible 
for the evaluation of their local programs. 

The major data collection instrument employed in the evaluation 
of the program has been the BST results. Student level means on the 
language arts and mathematics components of the test along with skill 
level diagnostic/prescriptive data have been the primary measure of 
program impact on student performance. Additionally, mul ti-copy 
Student Profile Sheets identifying each student's skill deficiencies 
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and requesting background information for each child have been dis- 
seminated to local school systems. These provide compensatory/ 
remedial teachers with a vehicle for monitoring student progress, and, 
at the same time they also serve as a technique for State level data 
collection relative to each student served by the program. One copy 
of the form is forwarded to the Department of Education at the end of 
the summer session, a second updated copy is submitted at the end of 
the regular school year, the third is retained as a central office 
copy, and the original remains in the student's cumulative folder. 

The constraints imposed by the nonavailability of pretreatment- 
posttreatment data have been addressed through the development of 
Compensatory Education Tests (CETs) that parallel each grade level 
BST. Through the use of these tests the impact of the program on 
students who are promoted, and thus take the next level BST after 
receiving services, can be determined. The CETs have been 
administered to samples of summer school participants in July and to 
regular year students in January/February so that the test results can 
be made available to the Legislative committees in March/April. The 
administration of the CETs also provides a vehicle for comparing t\\a 
performance of the promoted and retained students who failed a 
particular grade level BST. 

The evaluation results for the 1983-84 State-Funded Compensatory/ 
Remedial Program were reported through the use of a modular format. 
In this manner, evaluative information relative to each topic ad- 
dressed could be effectively and efficiently presented to the audi- 
ences of record ^or use within their required timelines. The publi- 
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cation dates and topics addressed by each report are listed below: 



1983-84 REPORTING STRATEGIES 



TIME 



TOPIC 



October 83 
November 83 
February 84 

March 84 
May 84 
June 84 
August 84 
September 84 



Student Placement Study 

1983 Summer School Evaluation 

Retention vs. Retention and 
Remediation 

Program Issues 

Preliminary Program Effects 

Program Management 

Program Formats 

Final Evaluation 



LONGITUDINAL PPOGRaII EFFECTS 

Through the use of the various data col lection and reporting 
strategies previously discussed, the following results were presented 
in the final evaluation of the 1983-84 State-Funded Compensatory/ 
Remedial Program (September 1984). The report drew primarily on data 
from the 1982, 1983, and 1984 administrations of the Basic Skills 
Tests (BSTs) to provide longitudinal information concerning the 
effects of the program on student performance. 

The results indicated that in the two years since the Grade 2 BST 
was first administered, means scores in both language arts and 
mathematics have improved slightly. Similar improvements were noted 
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in both areas on the second year of testing with the Grade 3 BST. 

The improvement in BST scores led to a subsequent decrease in the 
number of students qualifying for compensatory/ remedial services since 
the initial 1982 testing. At the same time, the average number of 
identified skill deficiencies among service qualifiers dropped. 

The study indicated that students who failed to attain the 
minimum performance standard on a particular level BST, and who 
received compensatory/ remedial services while being retained at that 
level, performed well when retested on the same BST. (An earlier 
study had found that the combination of retention and remediation was 
more effective than retention alone in terms of subsequent BST per- 
formance.) However, students who were promoted, and thus received 
compensatory/remedial services on lower level skills, did not perform 
well on the higher level BST. 

The examination of longitudinal data for the group of compensa- 
tory/remedial students who initially qualified for services on the 
1982 Grade 2 BST indicated that approximately two-thirds of these 
students had been retained once during the 1982-83 period. The 
performance of these students on the 1984 Grade 3 BST suggested that 
retention at grade 2 may be more cost effective than retention at 
grade 3 because, while 1984 scores were comparable, fewer remediation 
units had been required among the participants who had been retained 
at the earlier grade level. 

Two recommendations were made as a result of this study. The 
data supported the continued focus of the State-Funded Compensatory/ 
Remedial Program on identified deficit skills but suggested that this 
focus be broadened to address additional skills prerequisite to the 
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next grade level BST. This was viewed as particularly crucial for 
students who were promoted and had to take the next grade level BST 
after having received services in the lower level skills. Secondly, 
school systems were urged to consider very carefully the decision to 
promote students who fail to attain the minimum performance standard 
on a specific BST; retention at the earliest level at which deficien- 
cies are identified was strongly recommended. 

SUMMARY 

The specific data collection and reporting constraints discussed 
in this paper relative to Louisiana's State-Funded Compensatory/ 
Remedial Program, along with the evaluation strategies that have 
subsequently evolved to address these, provide an excellent illu- 
stration of the emerging State Education Agency (SEA) mode of ope- 
ration in the reporting of evaluative data to their respective Legis- 
latures. In spite of conflicting calendars, and in view of stringent 
time constraints, SEAs have adapted their reporting procedures to 
insure that the most effective and efficient use will be made of 
evaluation results. Not only the Legislature, but the State as a 
whole has benefited. 
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